Robust linear classification from limited training data

We consider the problem of linear classification under general loss functions in limited-data setting. Overfitting is a common here. The standard approaches to prevent overfitting are dimensionality reduction and regularization. But loses information, while regularization requires user choose norm, or prior, distance metric. propose an algorithm called RoLin that needs no choice applies large class functions. combines “reliable” information from top principal components with robust optimization extract any useful “unreliable” subspaces. It also includes new cross-validation better than existing methods Experiments on 25 real-world datasets three show broadly outperforms both Dimensionality has $$14\%-40\%$$ worse test average as compared RoLin. Against $$L_1$$ $$L_2$$ regularization, can be up 3x for logistic 12x squared hinge loss. differences greatest small sample sizes, where achieves best 2x more competing method. For some datasets, 15 training samples norm-based 1500 samples.